Tootfinder

Learning Structural Causal Models through Deep Generative Models: Methods, Guarantees, and Challenges
Audrey Poinsot, Alessandro Leite, Nicolas Chesneau, Mich\`ele S\'ebag, Marc Schoenauer
https://arxiv.org/abs/2405.05025

Learning Structural Causal Models through Deep Generative Models: Methods, Guarantees, and Challenges
This paper provides a comprehensive review of deep structural causal models (DSCMs), particularly focusing on their ability to answer counterfactual queries using observational data within known causal structures. It delves into the characteristics of DSCMs by analyzing the hypotheses, guarantees, and applications inherent to the underlying deep learning components and structural causal models, fostering a finer understanding of their capabilities and limitations in addressing different counter…

This https://arxiv.org/abs/2405.01649 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
Answering complex logical queries over incomplete knowledge graphs (KGs) is challenging. Most previous works have focused on learning entity/relation embeddings and simulating first-order logic operators with various neural networks. However, they are bottlenecked by the inability to share world knowledge to improve logical reasoning, thus resulting in suboptimal performance. In this paper, we propose a complex logical reasoning schema over knowledge graphs upon large language models (LLMs), co…

This https://arxiv.org/abs/2404.03194 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDB_…

Reservoir Sampling over Joins
Sampling over joins is a fundamental task in large-scale data analytics. Instead of computing the full join results, which could be massive, a uniform sample of the join results would suffice for many purposes, such as answering analytical queries or training machine learning models. In this paper, we study the problem of how to maintain a random sample over joins while the tuples are streaming in. Without the join, this problem can be solved by some simple and classical reservoir sampling algo…

Semi-Supervised Learning for Deep Causal Generative Models
Yasin Ibrahim, Hermione Warr, Konstantinos Kamnitsas
https://arxiv.org/abs/2403.18717 https://…

Semi-Supervised Learning for Deep Causal Generative Models
Developing models that can answer questions of the form "How would $x$ change if $y$ had been $z$?" is fundamental for advancing medical image analysis. Training causal generative models that address such counterfactual questions, though, currently requires that all relevant variables have been observed and that corresponding labels are available in training data. However, clinical data may not have complete records for all patients and state of the art causal generative models are unable to ta…

This https://arxiv.org/abs/2403.00867 has been replaced.
link: https://scholar.google.com/scholar?q=a

Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes
Large Language Models (LLMs) are becoming a prominent generative AI tool, where the user enters a query and the LLM generates an answer. To reduce harm and misuse, efforts have been made to align these LLMs to human values using advanced training techniques such as Reinforcement Learning from Human Feedback (RLHF). However, recent studies have highlighted the vulnerability of LLMs to adversarial jailbreak attempts aiming at subverting the embedded safety guardrails. To address this challenge, t…

This https://arxiv.org/abs/2405.01649 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
Answering complex logical queries over incomplete knowledge graphs (KGs) is challenging. Most previous works have focused on learning entity/relation embeddings and simulating first-order logic operators with various neural networks. However, they are bottlenecked by the inability to share world knowledge to improve logical reasoning, thus resulting in suboptimal performance. In this paper, we propose a complex logical reasoning schema over knowledge graphs upon large language models (LLMs), co…

Understanding LLMs Requires More Than Statistical Generalization
Patrik Reizinger, Szilvia Ujv\'ary, Anna M\'esz\'aros, Anna Kerekes, Wieland Brendel, Ferenc Husz\'ar
https://arxiv.org/abs/2405.01964

Understanding LLMs Requires More Than Statistical Generalization
The last decade has seen blossoming research in deep learning theory attempting to answer, "Why does deep learning generalize?" A powerful shift in perspective precipitated this progress: the study of overparametrized models in the interpolation regime. In this paper, we argue that another perspective shift is due, since some of the desirable qualities of LLMs are not a consequence of good statistical generalization and require a separate theoretical explanation. Our core argument relies on the…

Dynamically Anchored Prompting for Task-Imbalanced Continual Learning
Chenxing Hong, Yan Jin, Zhiqi Kang, Yizhou Chen, Mengke Li, Yang Lu, Hanzi Wang
https://arxiv.org/abs/2404.14721

Dynamically Anchored Prompting for Task-Imbalanced Continual Learning
Existing continual learning literature relies heavily on a strong assumption that tasks arrive with a balanced data stream, which is often unrealistic in real-world applications. In this work, we explore task-imbalanced continual learning (TICL) scenarios where the distribution of task data is non-uniform across the whole learning process. We find that imbalanced tasks significantly challenge the capability of models to control the trade-off between stability and plasticity from the perspective…

Generating, Reconstructing, and Representing Discrete and Continuous Data: Generalized Diffusion with Learnable Encoding-Decoding
Guangyi Liu, Yu Wang, Zeyu Feng, Qiyu Wu, Liping Tang, Yuan Gao, Zhen Li, Shuguang Cui, Julian McAuley, Eric P. Xing, Zichao Yang, Zhiting Hu
https://arxiv.org/abs/2402.19009

Generating, Reconstructing, and Representing Discrete and Continuous Data: Generalized Diffusion with Learnable Encoding-Decoding
The vast applications of deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations -- across various data types, such as discrete text/protein sequences and continuous images. Existing model families, like Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), autoregressive models, and diffusion models, generally excel in specific capabilities and data types but fall short in others.…

This https://arxiv.org/abs/2403.15268 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Imagination Augmented Generation: Learning to Imagine Richer Context for Question Answering over Large Language Models
Retrieval-Augmented-Generation and Gener-ation-Augmented-Generation have been proposed to enhance the knowledge required for question answering over Large Language Models (LLMs). However, the former depends on external resources, and both require incorporating the explicit documents into the context, which results in longer contexts that lead to more resource consumption. Recent works indicate that LLMs have modeled rich knowledge, albeit not effectively triggered or activated. Inspired by this…